Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update release flow #251

Merged
merged 2 commits into from
Jul 30, 2020
Merged

Update release flow #251

merged 2 commits into from
Jul 30, 2020

Conversation

clrcrl
Copy link
Contributor

@clrcrl clrcrl commented Jun 30, 2020

Description & motivation

As per our discussions — fixing a process problem with documentation rather than fancy tech.

🚨 We need to update our default branches to get this right.

@clrcrl clrcrl changed the base branch from master to dev/0.17.0 June 30, 2020 21:18
@clrcrl clrcrl requested a review from jtcohen6 June 30, 2020 21:18
@clrcrl clrcrl marked this pull request as draft July 9, 2020 19:45
@clrcrl clrcrl changed the base branch from dev/0.17.0 to dev/0.18.0 July 9, 2020 19:48
@clrcrl clrcrl force-pushed the update-release-flow branch 3 times, most recently from 82cc23b to abc5555 Compare July 9, 2020 20:01
@clrcrl clrcrl changed the base branch from dev/0.18.0 to dev/0.18.x July 9, 2020 20:20
run_test.sh Show resolved Hide resolved
@clrcrl clrcrl marked this pull request as ready for review July 9, 2020 20:31
Copy link
Contributor

@jtcohen6 jtcohen6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[EDIT: overtaken by events]

While the version of the world where all dbt and dbt-utils minor releases match their numbering, and all dependent packages defensively place upper boundaries on both, it introduces more complexity than we have capacity to handle proactively today. Instead, we're going to continue to let dbt and dbt-utils minor versions diverge, and react swiftly to breaking changes as they relate to each other and dependent packages.

Below is my original message for posterity, or for when we code up dbt-dependabot:

My feelings have gone back and forth on this several times. At the end of the day, I think it is the right move. There are two big preconditions for this to work well:

  • For every minor version release of dbt, we need to have a dbt-utils release ready to go on the same day ("gameday").
  • What to do about all our other packages? I don't think we need to move them into semantic version lockstep just yet. I think those packages should have the freedom to make minor releases (breaking changes) separate from the dbt release cycle. That said, we should pin versions defensively by adding the identical upper bounds for dbt + dbt-utils.

Logistics

The Core dev team is committing to putting out a release candidate two weeks before minor versions are released. In that crucial two-week window, we need to:

  • Test and ready the matching minor version of dbt-utils, for release on gameday.
  • If there are no breaking changes, cut a patch release of every other (non-utils) package to bump up the upper bounds of dbt + dbt-utils by one minor version. If there are breaking changes, prepare new minor versions of those packages to release on gameday.

Next steps

This is kind of messy, because we don't have a dev/0.17.x branch that people should make bug-fix PRs against.

What do you think about taking these steps now:

  • Create a dev/0.17.x branch off master, and make it the default branch
  • Cut a 0.17.0 release that's effectively identical to 0.5.0
  • Cut new patch release of every other Fishtown-supported package to pin all three of the following at ">=0.17.0,<0.18.0":
    • require-dbt-version
    • dbt-utils revision
    • pip install dbt into

0.18.0

We hope to have a release candidate of dbt v0.18.0 by the end of next week.

  • Should we cut a dbt-utils v0.18.0-rc1, pre-gameday?
  • Or is it enough to install dbt-utils from the dev/0.18.x branch when testing other packages that depend on it?

If it turns out that dbt v0.18.0 and dbt-utils v0.18.0 don't have any breaking changes for (e.g.) the Snowplow package, we can cut a new patch release of the Snowplow package right then and there, pre-gameday, just to update its dependency boundaries to ">=0.17.0,<0.19.0".

@clrcrl
Copy link
Contributor Author

clrcrl commented Jul 30, 2020

@jtcohen6 — I rewrote almost all of this and just did a force push. I think there's probably still some details to hash out once you've read through it, especially w.r.t. branching strategy.

@clrcrl clrcrl changed the base branch from dev/0.18.x to main July 30, 2020 19:46
Copy link
Contributor

@jtcohen6 jtcohen6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for making something (a saner, proportional-to-our-efforts approach) out of nothing (my contributions to a meandering discussion this morning).

I think the biggest trick is going to be adjudicating what counts as "new functionality." New macros makes sense to me. What about (e.g.) adding a new argument to the pivot macro, with no breaking changes for existing users?

IMO we want to be in the habit of releasing dbt-utils minor versions less often than dbt minor versions, since there's a lot of dependency wrangling any time we do it. That may mean making net-new contributors wait longer than they've had to in the past.

## Branching strategy

At any point, there should be two long-lived branches:
- `main`: This reflects the most recent release of dbt-utils
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking, "Shouldn't this be dev/0.x-1.0 (currently dev/0.5.0)? But I think having a safe, clear, main branch to PR against for small fixes or README updates is going to be much more user-friendly.

@clrcrl clrcrl merged commit 75587ed into main Jul 30, 2020
@clrcrl clrcrl deleted the update-release-flow branch July 30, 2020 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants